Skip to content

Increase timeout for check_rule_active from 60s to 120s#22260

Merged
abdosi merged 2 commits intosonic-net:masterfrom
arista-nwolfe:master-everflow-ipv6-rule-timeout-increase
Feb 18, 2026
Merged

Increase timeout for check_rule_active from 60s to 120s#22260
abdosi merged 2 commits intosonic-net:masterfrom
arista-nwolfe:master-everflow-ipv6-rule-timeout-increase

Conversation

@arista-nwolfe
Copy link
Contributor

@arista-nwolfe arista-nwolfe commented Feb 5, 2026

We've seen that acl rule installation can take a while as discussed in #21988

This check_rule_active/check_rule_counters code was added in #21384
but it never worked on DNX due to unsupported SAI calls: sonic-net/sonic-buildimage#24665

Now that we're working around this issue with:
#22237
We're starting to run into timeouts waiting for ACL rules: Failed: Acl rule counters are not ready

Here are the logs from a failing run where you can see the time it takes for the last ACL rule to be installed takes more than 60s:

2026 Feb  5 03:44:23.106324 ctn101 INFO python[509061]: ansible-ansible.legacy.command Invoked with _raw_params=acl-loader update full /tmp/everflow/acl-erspan.json --table_name EVERFLOW_EGRESSV6 --session_name test_session_1 --mirror_stage egress
2026 Feb  5 03:44:27.267227 ctn101 INFO python[509601]: ansible-ansible.legacy.command Invoked with _raw_params=config load -y /tmp/everflow/acl_config.json

...

2026 Feb  5 03:46:22.380011 ctn101 NOTICE swss#orchagent: :- add: Successfully created ACL rule RULE_9 in table EVERFLOW_EGRESSV6

We're also increasing the timeout from 60s to 120s for the acl rules in the acl/ package tests here:
#21760

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • New Test case
    • Skipped for non-supported platforms
  • Test case improvement

Back port request

  • 202205
  • 202305
  • 202311
  • 202405
  • 202411
  • 202505
  • 202511

@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

We've seen that acl rule installation can take a while as discussed in
sonic-net#21988

This check_rule_active/check_rule_counters code was added in
sonic-net#21384
but it never worked on DNX due to unsupported SAI calls:
sonic-net/sonic-buildimage#24665

Now that we're working around this issue with:
sonic-net#22237
We're starting to run into timeouts waiting for ACL rules:
Failed: Acl rule counters are not ready

Signed-off-by: Nathan Wolfe <[email protected]>
@arista-nwolfe arista-nwolfe force-pushed the master-everflow-ipv6-rule-timeout-increase branch from 9c8ab96 to db1856b Compare February 5, 2026 19:35
@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.


if duthost.facts['asic_type'] != 'vs':
pytest_assert(wait_until(60, 2, 0, self.check_rule_active, duthost, table_name),
pytest_assert(wait_until(120, 2, 0, self.check_rule_active, duthost, table_name),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I had trouble getting the test pass even at 120s consistently, so I had to increase this to 150s to get it to pass. Are you seeing consistent success with q3d with this timeout?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've only ran it a handful of times but all passed. I'm fine with bumping this to 150s as I was also seeing the timings from the logs were pretty close to 120s.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to 150s

@mssonicbld
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@abdosi
Copy link
Contributor

abdosi commented Feb 18, 2026

@arista-nwolfe : Please create Issue with slowness of rule insertion

@abdosi abdosi merged commit 90cc07c into sonic-net:master Feb 18, 2026
18 checks passed
anilal-amd pushed a commit to anilal-amd/anilal-forked-sonic-mgmt that referenced this pull request Feb 19, 2026
…22260)

* Increase timeout for check_rule_active from 60s to 120s

We've seen that acl rule installation can take a while as discussed in
sonic-net#21988

This check_rule_active/check_rule_counters code was added in
sonic-net#21384
but it never worked on DNX due to unsupported SAI calls:
sonic-net/sonic-buildimage#24665

Now that we're working around this issue with:
sonic-net#22237
We're starting to run into timeouts waiting for ACL rules:
Failed: Acl rule counters are not ready

Signed-off-by: Nathan Wolfe <[email protected]>

* Bump up from 120s to 150s as timeouts are still seen

Signed-off-by: Nathan Wolfe <[email protected]>

---------

Signed-off-by: Nathan Wolfe <[email protected]>
Signed-off-by: Zhuohui Tan <[email protected]>
@arista-nwolfe
Copy link
Contributor Author

@arista-nwolfe : Please create Issue with slowness of rule insertion

Requested issue: sonic-net/sonic-buildimage#25565

aronovic pushed a commit to aronovic/sonic-mgmt that referenced this pull request Mar 3, 2026
…22260)

* Increase timeout for check_rule_active from 60s to 120s

We've seen that acl rule installation can take a while as discussed in
sonic-net#21988

This check_rule_active/check_rule_counters code was added in
sonic-net#21384
but it never worked on DNX due to unsupported SAI calls:
sonic-net/sonic-buildimage#24665

Now that we're working around this issue with:
sonic-net#22237
We're starting to run into timeouts waiting for ACL rules:
Failed: Acl rule counters are not ready

Signed-off-by: Nathan Wolfe <[email protected]>

* Bump up from 120s to 150s as timeouts are still seen

Signed-off-by: Nathan Wolfe <[email protected]>

---------

Signed-off-by: Nathan Wolfe <[email protected]>
Signed-off-by: Mihut Aronovici <[email protected]>
@davidm-arista davidm-arista added the Request for 202511 branch Request to backport a change to 202511 branch label Mar 3, 2026
ravaliyel pushed a commit to ravaliyel/sonic-mgmt that referenced this pull request Mar 12, 2026
…22260)

* Increase timeout for check_rule_active from 60s to 120s

We've seen that acl rule installation can take a while as discussed in
sonic-net#21988

This check_rule_active/check_rule_counters code was added in
sonic-net#21384
but it never worked on DNX due to unsupported SAI calls:
sonic-net/sonic-buildimage#24665

Now that we're working around this issue with:
sonic-net#22237
We're starting to run into timeouts waiting for ACL rules:
Failed: Acl rule counters are not ready

Signed-off-by: Nathan Wolfe <[email protected]>

* Bump up from 120s to 150s as timeouts are still seen

Signed-off-by: Nathan Wolfe <[email protected]>

---------

Signed-off-by: Nathan Wolfe <[email protected]>
Signed-off-by: Ravali Yeluri (WIPRO LIMITED) <[email protected]>
abhishek-nexthop pushed a commit to nexthop-ai/sonic-mgmt that referenced this pull request Mar 17, 2026
…22260)

* Increase timeout for check_rule_active from 60s to 120s

We've seen that acl rule installation can take a while as discussed in
sonic-net#21988

This check_rule_active/check_rule_counters code was added in
sonic-net#21384
but it never worked on DNX due to unsupported SAI calls:
sonic-net/sonic-buildimage#24665

Now that we're working around this issue with:
sonic-net#22237
We're starting to run into timeouts waiting for ACL rules:
Failed: Acl rule counters are not ready

Signed-off-by: Nathan Wolfe <[email protected]>

* Bump up from 120s to 150s as timeouts are still seen

Signed-off-by: Nathan Wolfe <[email protected]>

---------

Signed-off-by: Nathan Wolfe <[email protected]>
Signed-off-by: Abhishek <[email protected]>
vrajeshe pushed a commit to vrajeshe/sonic-mgmt that referenced this pull request Mar 23, 2026
…22260)

* Increase timeout for check_rule_active from 60s to 120s

We've seen that acl rule installation can take a while as discussed in
sonic-net#21988

This check_rule_active/check_rule_counters code was added in
sonic-net#21384
but it never worked on DNX due to unsupported SAI calls:
sonic-net/sonic-buildimage#24665

Now that we're working around this issue with:
sonic-net#22237
We're starting to run into timeouts waiting for ACL rules:
Failed: Acl rule counters are not ready

Signed-off-by: Nathan Wolfe <[email protected]>

* Bump up from 120s to 150s as timeouts are still seen

Signed-off-by: Nathan Wolfe <[email protected]>

---------

Signed-off-by: Nathan Wolfe <[email protected]>
Signed-off-by: Venkata Gouri Rajesh Etla <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Request for 202511 branch Request to backport a change to 202511 branch

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants